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ABSTRACT 


The  Protection  Analysis  project  was  initiated  at  ISI  by 


ARPA  IPTO  to 
vulnerabilities 
techniques  for 
system  software, 
make  protection 


further  understand  operating  system  security 
and,  where  possible,  identify  automatable 
detecting  such  vulnerabilities  in  existing 
The  primary  goal  of  the  project  was  to 
evaluation  both  more  effective  and  more 


economical  by  decomposing  it  into  more  manageable  and 
methodical  subtasks  so  as  to  drastically  reduce  the 
requirement  for  protection  expertise  and  make  it  as 
independent  as  possible  of  the  skills  and  motivation  of  the 
actual,  individuals  involved.  The  project  focused  on 
near-term  solutions  to  the  problem  of  improving  the  security 
of  existing  and  future  operating  systems  in  an  attempt  to 
have  some  impact  on  the  security  of  the  systems  which  would 
be  in  use  over  the  next  ten  years, 


A general  strategy  was  identified,  referred  to  as 
"pattern-directed  protection  evaluation"  and  tailored  to  the 
problem  of  evaluating  existing  systems.  The  approach 
provided  a basis  for  categorizing  protection  errors 
according  to  their  security-relevant  properties;  it  was 
successfully  applied  for  one  such  category  to  the  MULTICS 
operating  system,  resulting  in  the  detection  of  previously 
unknown  security  vulnerabilities. 
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The  Protection  Analysts  project  was  initiated  at  ISI  by  ARPA  IPTO  to  further 
understand  operating  system  security  vulnerabilities  and,  where  possible,  identify 
automatable  techniques  for  detecting  such  vulnerabilities  in  existing  system  software.  The 
primary  goal  of  the  project  was  to  make  protection  evaluation  both  more  effective  and 
more  economical  by  decomposing  it  into  more  manageable  and  methodical  subtasks  so  as  to 
drastically  reduce  the  requirement  for  protection  expertise  and  make  it  as  irKlependent  as 
possible  of  the  skills  and  motivation  of  the  actual  individuals  involved.  The  project  focused 
on  nn  ir-fcrm  solutions  lo  the  problem  of  improving  the  security  of  existing  and  future 
operaling  systems  in  an  attempt  to  have  some  impact  on  the  security  of  the  systems  which 
would  be  in  use  over  the  next  ten  years. 

A general  strategy  was  identified,  referred  to  as  "pattern-directed  protection 
evaluation"  and  tailored  to  the  problem  of  evaluating  existing  systems.  The  approach 
provided  a basis  for  c.atogor!2ing  protection  errors  according  to  their  security-relevant 
properties;  it  was  successfully  applied  for  one  such  category  to  the  MULTICS  operating 
system,  resulting  in  the  detection  of  previously  unknown  security  vulnerabilities. 
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WiK'n  f.t'MPral  purpose  resource-sharing  operating  systems  became  available, 
system  cuslomors  (both  governmental  agencies  and  private  firms)  naturally  wished  to 
exploit  fully  the  economies  such  systems  offered  in  processing  sensitive  together  with 
nonsensilive  information.  Responding  to  customers'  pressure,  the  systems’ 
manufacturers  at  first  claimed  that  the  hardware  and  software  mechanisms  supporting 
resource  sharing  would  also  (with  perhaps  minor  alterations)  provide  sufficient 
protection  anc<  isolation  lo  permit  multiprogramming  of  sensitive  and  nonsensitive 
programs  and  data.  A skeptical  technical  community  challenged  this  claim  and  proved  it 
false.  Relatively  cursory  inspection  of  selecleci  operating  systems  by  "tiger  teams" 
(individuals  brought  together  specifically  to  atfeaipt  to  penetrate  a target  operating 
system)  established  that  the  protection  offered  fell  far  short  of  that  required  if 
muMiprogramming  of  sensitive  and  nonsensilive  programs  and  information  were  to  be 
permitted  fAnd-*?!,  Rran73).  The  protection  mechanisms  functioned  adequately  wherr 
users  exercised  proscribed  system  furtetions  in  approximately  the  prescribed  way,  but 
could  not  resist  the  system  penetrator  who  looked  for  unusual  or  extraordinary  means 
to  avoid  access  checking. 

lacking  some  of  today's  insight  and  knowledge,  various  manufacturers  attempted 
to  retrofit  their  existing  operating  systems  for  security  by  simply  correcting  the 
individual  implomcnialion  errors  and  obvious  design  oversights  that  contributed  to  their 
system’s  security  deficiencies.  Critical  analysis  of  these  systems,  however,  established 
that  piecemeal  efforts  lo  secure  an  existing  general-purpose  operating  system  were 
unlikely  to  succeed  (Abb  + 76,  Afl  + 76,  BelW/A,  MolG7A,  Mcph7A] 

Out  of  this  e.arly  floundering  came  an  appreciation  that  the  security  problem  was 
mud)  more  difficult  lo  deal  with  than  expected,  furthermore,  a number  of  disturbing 
issues  surfaced: 

I.  The  question  Of  what  conslilutcd  an  appropriate  degree  of  security  and  how 
till'.  IS  <l<'lermined  for  a computer  system  had  not  been  adequately  addressed. 
Indeed,  ttie  notion  of  security  was  itself  difficult  to  formalize  in  the  context  of 
computer  systems,  i.c.,  it  was  a rcsearcli  issue  in  its  own  rigfit.  Intuitive 
slalemenls  such  as  "the  system  should  not  allow  an  unauthorized  user  to 
access  information  he  had  no  right  lo  access"  somehow  had  to  be  translated 
into  spc'cific  assertions  about  specific  operating  system  objects. 

?.  No  methodology  existed  for  insuring  that  a given  system’s  design  was 
complete  with  respect  to  a particular  security  policy  which  might  be  chosen, 
i.e.,  that  there  wore  not  substantial  or  significant  areas  where  the  desired 
protection  policy  could  simply  bo  circumventcff  or  Ignored. 

3.  existing  operating  systems  were  pooHy  structured  when  it  came  to  security 
and  integrity,  usually  having  grown  from  early  releases  tc  patched, 
error -ridden  monoliths  of  interconnected  code  and  tables. 
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4.  Efforts  to  correct  known  errors  were  e$  likely  as  not  to  introduce  an  equal 
number  of  new  errors,  merely  manifested  In  other  ways,  This  became 
painfully  evident  during  the  system  penetration  activities  conducted  In 
conjunction  with  security  retrofit  efforts. 

5.  Progran)  verification  techniques  would  ultimately  have  to  be  applied  to  insure 
that  operating  system  code  functioned  correctly  and  according  to 
specification.  However,  existing  techniques  could  handle  only  relatively  small 
pieces  of  code,  limited  data  types,  and  relatively  simple  data  structures  and 
data  accessing  schemes--nothing  within  an  order  of  magnitude  Of  the  size  and 
complexity  of  an  operating  system  as  then  structured  and  implemented. 

While  those  and  other  issues  were  troublesome  enough  with  regard  to  future 
systems,  they  were  particularly  troublesome  in  light  of  the  large  inventory  of  systems 
In  the  DoO  and  private  sector.  It  had  been  suggested  that  an  existing  operating  system 
would  have  to  bo  restructured  if  any  substantial  improvement  in  the  security  afforded 
was  to  be  effected  or  if  program  verification  techniques  were  to  be  successfutly 
applied.  However,  rostructuring  of  an  existing  system  (in  many  cases  tantamount  to 
redesign  of  the  system)  meant  committing  substantial  resources  and  rewriting  a 
considerable  amouni  of  code.  It  was  also  apparent  that  this  could  be  considered  only 
for  a few  special  systems  such  as  MULTICS  and  VM/370,  which  were  already 
wcll-sfrucfured  with  the  access  control  mechanisms  at  the  innermost  level  of  control. 

It  became  obvious  that  additional  insight  into  the  design  and  implementation 
deficiencies  responsible  for  operating  system  security  vulnorabililies  was  necessary.  A 
much  more  comprehensive  view  was  required  of  the  number  and  form  taken  by  such 
vulnerabilities.  The  system  penetration  work  performed  in  the  past  did  tittle  to  provide 
any  such  collective  insight,  however;  the  expertise  resulting  from  such  studies  consisted 
of  the  individual  insights  of  a few  individuals  rather  than  communicable  Ideas  end 
knowledge. 
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2.  rROJHCr  DKSCRIPIION  /)NI)  Asr}KnriONs 


In  Soptcinhcr  ol  1973,  the  Ptoleclion  Analysis  projoci  was  initiated  at  ISI  by 
ARPA  IPIO  to  onhanrc  Our  understanding  of  operating  system  vulrmrsbilllies,  expar\d  the 
rather  sparse  knowicdi’o  base  on  Ibis  subject,  and,  if  possible,  identify  automatable 
tochtriquos  for  (Ir  lrrlinp.  viilnei  abilities  in  existing  system  software.  Near-term 
solutions  to  the  prol'lem  of  improving  tire  security  ol  oxisting  and  future  systems  wore 
important  if  operating  systems  security  research  was  to  have  much  Impact  on  the 
eystems  which  would  be  in  use  over  the  next  ten  yoars.  It  was  hoped  that  the  effort 
would  yield  a more  formalized  knowledge  base  on  operating  system  security,  making  It 
possible  to  decouple  security  and  oporatmg  sysleai  expertise  to  some  degree,  I.e.,  to 
allow  individuals  having  limited  expertise  in  operating  system  security  to  effectively 
detect  System  vulnerabilities, 

lh('  approach  adopted  was  a significant  departure  from  the  protection  evaluation 
projects  going  on  eisewheie  at  that  lime,  such  as  those  at  Project  RISOS  and  at  System 
Ofvclopmcnl  Corporation.  These  oflorls  to  syslemnnze  penciralion  activities  dealt 
primarily  with  llie  oig.-iriiaation  of  fhc  proieci  staff  itself  rather  tlian  tlio  discipline 
applied  (Wcis/31.  Ilu'y  .addressed  tlio  orgnnizaliotinl  and  training  aspects  Of  teams  of 
individuals  tasked  to  analyze  operating  systems  for  security  vulnorabilitios--individuals 
who  ther)iselves  would  m.'iKe  good  "penotralors"  of  a given  targe!  sysism,  who  had  not 
only  an  jnlirnate  knowlodgo  of  that  system  but  also  a good  under  standing  of  and  fool  for 
protcclion  error  po'.s-ibililies. 

II  was  evident  that  Iho  success  ol  such  r.ioiijjs  would  depend  he.avily  on  individual 
motivation  as  well  as  skill  in  finding  proleclion  errorE--nn  apparent  shortcoming  when  it 
came  to  aiaking  dcdmilive  statements  about  the  validity  ol  Iho  evaluation  effort  in  which 
such  an  aiipro.icti  was  adopted  The  primary  goal  ol  the  151  project  was  to  rnake 
protection  cvahialion  both  more  effective  and  more  economical  by  decomposing  it  into 
more  nuinagc'abir  and  meltiodiral  sul>lasks  so  as  to  drastically  reduce  the  i'oquirement 
tor  proirction  experhso  and  make  if  as  ii'depondenl  as  possible  of  the  skills  and 
molivat'on  of  the  aclvi,il  individuals  irivolvod. 

A general  strategy  was  identified  which  promised  to  meet  these  objoctivos.  It 
included  the  following  five  steps: 

1.  Collection  ol  "raw"ciior  do'cnplions. 

?.  R('i  eprr'sr'ntation  of  raw  errc>r  descriptions  in  a more  formalized  notation 
(producinii  "raw  error  pattorns"). 

3.  riimin.ilion  of  ouperlluous  features  and  abstraction  of  specific  system 
elomenls  inlo  system-iriclGpendei't  elements  to  develop  generalized  error 
p.TlIern'^. 

A.  "Normalization"  of  the  fatfel  system  l>y  ovlracting  the  information  relevant  to 
ttic'  evalu.ition  ancf  represeni;-"  it  in  Iho  lortn  required  by  a "comparison" 
pt  oceduie 
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!).  Execution  of  the  comparison  procedure. 

TIvo  specific  approach  adopted— subsequenUy  referred  to  as  "pattern-directed 
protection  evaluation"  [Car+75]— was  tailored  to  the  problem  of  evaluating  existing 
systof>«.  It  differed  from  the  more  gonoral  approach  principally  In  that  apecitic 
features  of  interest  wore  "extracted*  from  the  operating  system  source  code  rather 
than  the  entire  operating  system  being  rorepresented  In  a "normallied"  format 
fFigure  1),  Thus,  stops  ^ and  5 changed  as  follows; 

4.  "Feature  extraction":  Instantiation  of  generalized  features  and  searches  for 
Instances  of  those  features  in  the  target  Operating  system,  and  the 
description  of  their  relevant  contexts. 

5.  Comparison  of  combinations  of  feature  instances  and  their  contexts  with  the 
loatures  and  relations  expressed  in  the  approprialo  error  patterns. 

A major  expectation  was  that  adopting  this  approach  would  make  it  easier  to 
identify  previously  undiagnosed  errors  in  given  operating  systems.  As  superfluous 
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(t'alures  and  qualifying  dofails  were  eliminated  and  specific  system  features  roptaeect  by 
more  Rcnerlc  or  abstract  features,  a more  Benorallzed  error  representation  would 
evolve.  The  process  could  conceivably  result  in  a hierarchy  Of  error  patterns,  wHtr  tlie 
most  general  and  abstractly  dolinotl  patterns  at  tlio  upper  levels  and  the  most 
specialized  and  concrete  ones  at  the  lower  levels.  Subsequent  Inslantiation  of  the 
generalized  paltorns  by  roplacmu  the  more  fioneral  features  with  thoir  more  specific 
counterparts  In  particular  classes  of  operating  systems  or  particular  functional  areas 
rnighf  be  expoclod  to  reveal  pisviously  undiscovered  Operating  system  errors 
(Fipurc  S). 


Major 
error  types 

O 

Build  cotegorles 
from  error  analysis 

...  generollzed  patterns  ... 

/\ 

row  error  potterns  , , . 


, , , errors  , , , errors  , , , errors  , , , 


Lattice  of  error  pattoins 
Error  search  procedures 
New  errors  identified 


Figure  2 


A second  cNpcclalion  was  that  this  approach  minhl  result  In  an  empirically  sound 
taxonomy  of  oporaling  system  vulnorabilitios  and  their  causes,  wliicli  would  bo 
particularly  useful  for  system  designers  and  implerr.anlors.  The  derivation  of  raw 
patter  ns,  Ihoir  generalization,  and  the  irislantialion  of  generalized  patterns  toward  other 
systems  and  functional  areas  would  all  add  new  olomonis  to  the  lattice  of  patterns 
formed  by  Itic  relation  ''per\oralizalion  of"  and  its  corweise,  'instance  of,"  witfi  the  more 
abstract  patterns  al  the  fop  and  the  more  concrete  ones  at  the  bottom.  As  this 
structure  was  enriched  with  additional  patterns,  major  substructures  might  emerge,  at 
least  below  some  level  of  abstractrwss.  If,  as  was  also  expected,  the  search  techniques 
dolerminod  to  be  aopropriate  for  the  patfeins  of  each  such  substructure  were  also 
similar,  then  a reasonable  basis  would  be  provided  to  define  major  "error  types." 

The  approach  was  tested  with  regard  to  a part  cule'  error  type  frequently  found 
in  operating  systems,  and  it  proved  successful  at  uncovering  previously  undlagr^osod 
errors  in  the  MULTICS  operating  system  tOis+75,  Ois+76].  The  specific  details  of  the 
approach  and  the  results  and  problems  which  ensued  are  discussed  in  the  sections 
which  follow. 
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couHcrtoN  or  h/w  hhhoh  nnr/i 

Prior  to  Ihir.  projoct,  little  <tnta  oi'  known  protection  error  vulnerebllllles  hed 
sctiinlly  boon  assomblod  as  such  in  one  plate.  Thus,  the  first  phase  of  the  project 
Involved  dovclopinp,  a sufficiently  rich  collection  of  data  on  operating  syitem  error* 
from  ns  many  oporatinR  systems  ns  possible  to  provide  a good  sampling  Of  the  types  of 
errors  which  existed. 

Ullimololy  more  than  100  errors  that  could  be  employed  directly  to  penetrate 
existinp  oporalinp,  systems  wore  recorded  in  an  error  data  base)  numerous  minor 
varinliona  on  these  errors  wore  also  possible.  These  errors  came  from  six  syslemsi 
TIiNrx,  MULTICS,  t Xte  8.  GCOS,  UNIX,  and  OS/360. 

The  prO]erl  stnfl  itsolf  was  familiar  in  varying  degrees  with  fivo  Of  the  six 
operating  systems  They  had  boon  diroclly  involved  in  penetration  work  on  only  three 
of  these  operating  systems,  however,  anc<  then  in  projects  which  examined  the  systems 
at  widely  diflering  levels  ol  detail.  Consociuontly,  the  project  had  to  rely  to  some 
extent  upon  information  it  could  gathor  from  outsido  sources,  namely  Other  Individuals 
involved  in  operating  system  penolralion  studies. 

Unlorlunalely,  it  was  difficult  <0  acquire  useful  data  on  errors  for  systems  which 
had  not  been  direclly  revlowtid  by  the  stall.  Perhaps  the  major  difficulty  was  the 
unavailability  of  .my  overall  information  about  operating  system  vulnerabilities, 
principally  because  most  inslallahons  were  •■eluftani  to  air  weaknesses  that  might 
subsequently  bo  evpioitcd  by  individuals  insido  as  well  as  outsido  thoir  organizations. 
Ar\othcr  significant  diKiculty  also  arose  whose  principal  impact  was  felt  In  the 
development  of  raw  error  pattornsi  it  is  (liscus;»d  in  llie  following  faction. 


im'n.opMr.NV  or  «/iir  mnon  patvkkns 

(liven  a raw  error  description,  the  next  stop  w'as  to  formulate  an  appropriate  raw 
error  pnifern,  a rcdescription  of  the  error  in  terms  spcdhc  to  its  source  operating 
system  but  in  the  form  of  predicates  that  express  "conditions,"  properties  of  or 
relations  among  dishncf  objects  or  features  ol  that  system.  During  this  process  those 
aspects  of  the  initial  description  superfluous  to  the  actual  error  itself  wore  eliminated. 
The  "condition  set"  of  a raw  pattern  was  a minimal  sot  of  conditions  in  the  sense  that  If 
any  were  removed  the  raw  pattern  would  no  longer  represent  a potential  error, 

flowever,  from  a particular  raw  error  description,  it  was  often  extremely  difficult 
to  write  down  a pattern  that  salictnctorily  captured  the  essence  of  the  error,  rirst,  of 
coiirs^'.  the  error  description  had  to  be  thoroughly  co.mprehendod,  e.g..  In  terms  of  how 
the  error  could  be  exploited  by  .a  knowledgoablo  ponetrator  This  required  substantial 
familiarity  with  and  sufficient  information  on  the  operating  system  context  in  which  It 
occurred.  Unfortunately,  oven  where  $ucf>  information  was  available,  the  errors  were 
sometimes  describee)  m a rattier  incomololo  fashion  or  in  a fastiion  which  presumed 
substanlial  knowledge  about  specific  low-level  details  of  the  system  implementation. 
This  was  further  complicated  by  (he  lack  of  a common  vocabulary  for  describing  both 
functional  elements  of  the  system  as  well  as  the  particulars  of  a given  security 
deficiency,  requiring  some  conjecture  on  the  part  ol  the  staff  as  to  the  exact 
circumstances  of  the  problem. 
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Drr.pifo  thow  complications,  the  staff  generally  was  fairly  successful  In 
ascertaining  whal  appeared  to  be  the  significant  characteristics  of  the  error  from  the 
available  documentation.  Even  w'th  that,  however,  it  was  not  always  clear  precisely 
what  policy  was  being  violated  and  thus  what  conditions  should  constitute  the  pattern. 
In  some  cases,  in  which  equally  valid  policies  could  be  postulated,  the  same  raw  error 
appeared  to  lead  to  more  than  one  pattern. 

1his  process  did  not  appear  to  be  inordinately  difficult  in  the  case  of  the  first 
pattern  processed,  "Jnccnsistency  of  a Single  Data  Value  over  rime."  The  relevant 
characteristics  of  such  errors  were  readily  apparent,  as  manifested  in  the  various 
examples  in  the  error  data  base.  Thus,  the  textual  description  of  a given  instance  of 
the  error  type  was  successfully  rerepresented  in  a raw  pattern  for  which  superfluous 
details  had  been  eliminated.  This  is  illustrated  by  the  following  raw  error  description 
and  derived  raw  error  pattern  taken  from  an  early  version  of  MULTICS  [Bis+75]. 

Raw  Error  Dcsrriplion;  STOP- PROCESS-ERROR 

STOP-PROCESS  is  a supervisor  procedure  tor  halting  processes.  The  user  can  call  the 
procedure  with  the  process-id  of  the  process  to  be  slopped.  The  user  entry  to  this 
procedure  checks  that  the  ID  is  that  ol  the  caller,  then  calls  the  traffic  controller 
termination  routine.  The  user  can  modify  the  value  ol  the  process-id  between  the  time 
it  is  chocked  and  the  time  it  is  passed  to  the  traffic  controller. 

Raw  Error  Pattern: 

1.  Procedure  "STOP-PROCESS"  is  irivoked  by  a user  process  to  halt  a specified 
process  as  indicated  by  a user -supplied  parameter. 

2.  The  "STOP-PROCESS"  interface  checks  that  the  user-supplied  process-id 
parameter  is  valid. 

3.  The  traffic -coniroller  termination  routine  uses  the  process-id  to  identify  the 
appropriate  process. 

The  user  process  may  modify  the  checked  parameter  between  the  times  of  (2) 
and  (3). 


im'Ki.orMKNT  Oh  GhimnnuAKn  p/irrKKNs 

As  an  error  search  criterion,  a raw  pallern  is  directly  applicable  only  to  operating 
systems  that  share  the  policy  violated  by  that  error  and  in  which  the  features  of  that 
pattern  are  known  by  the  same  names.  Even  then,  it  may  apply  only  to  a particular 
functional  area  such  as  input/output  control,  and  miss  similar  errors  in  other  areas  such 
as  inIcrprcccGs  communication.  To  broaden  the  applicability  of  a pattern,  its  expression 
must  be  generalized  by  substituting  more  generic  names  or  more  abstract  features  for 
more  specific  ones  or  by  deleting  qualifying  details  without  affecting  the  essence  of  the 
conditions  themselves.  The  same  concept,  such  as  the  call  on  a privileged  system 
procedure  by  an  unprivileged  user  procedure,  may  be  known  by  different  names  (such 
as  "MME,"  "JSYS,"  and  "SVC")  in  different  systems.  Classes  of  similar  objects,  such  as 
f ytes  or  blocks  of  physical  storage,  pages,  segments,  variables,  structured  variables. 
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and  files  (to  give  an  extreme  example),  can  be  regarded  as  instances  of  • more  abstract 
object,  in  this  case  the  "abstract  cell,"  something  that  has  a name  and  holds  information 
(its  value).  The  benefit  of  generalizirg  is  that  the  generalized  pattern  applies  to  a 
correspondingly  wider  class  of  errors  in  a wider  class  of  systems. 

Gcr^cralization  of  the  raw  pattern  for  the  inconsistency  error  examples  yielded 
the  following  error  pattern  and  corresponding  security  policy  statement: 

(Tencratized  Error  Pattern: 

B:M(X)  and  lor  some  operation  L occurring  before 
[for  operation  L which  does  not  modify  Valuc(X), 

Value(X)  before  L NOT  - Value(X)  belore  M],  and 
Value(X)  alter  L NOT  = Value(X)  before  M. 

Informally  stated,  process  B performs  operation  M on  variable  X and  the  value  of  X at 
the  time  operation  K/f  is  performed  is  not  equal  to  the  value  of  X either  before  or  after 
some  operation  L which  occurs  before  M. 

Corresponding  Operating  System  Security  Policy  Statement: 

(0,M,X)  for  some  operation  L occurrir:g  before  M,  either 
[for  operation  L which  does  not  modify  Value(X), 

Value(X)  before  L » ValuefX)  before  M],  or 
Value(X)  after  L « Value(X)  before  M. 

Intuitively  stated,  process  B (which  presumably  performs  some  critical  function)  can 
perform  operation  M on  variable  X only  if  the  value  of  X at  the  time  operation  M is 
performed  is  equal  to  the  v.alue  of  X either  before  or  after  some  operation  L which 
occurs  before  M. 


h K/iWHK  KXTK/ICriOX 

[>rfccting  errors  in  a sot  of  target  information  implies  some  Kind  of  comparison 
process  between  the  target  and  the  correctness  or  error  criteria.  The  compar*son 
need  not  be  direct:  various  transformations  may  be  applied,  as  practical,  to  either  the 
criteria  or  the  target  to  bring  them  into  a suitable  form,  as  long  as  essential  properties 
are  preserved.  In  the  case  of  pattern-directed  piotcction  evaluation,  the  target  is  a set 
of  operating  system  source  programs  and  specifications;  the  criteria  are  the  error 
patterns;  and  the  comparison  process  is  essentially  one  of  "pattern  recognition,"  In  the 
sense  of  an  ability  to  delect  instances  of  errors  embedded  or  camouflaged  in  e system. 

Conceptually,  the  ideal  tool  is  a general-purpose  "protection  evaluator,"  s 
computer  program  that  not  only  could  be  applied  to  a wide  class  of  operating  systems 
but  could  also  reliably  detect  a wide  class  of  errors.  The  inputs  to  such  a program 
would  be  representations  of  the  patterns  for  the  error  types  covered,  together  with  a 
representation  of  the  target  operating  system.  The  program  would  compare  the  target 
representation  with  the  given  patterns  by  searching  it  for  all  combinations  of  features 
related  in  one  of  the  ways  specified  in  some  pattern,  and  would  report  every  such 
combination  found.  In  this  concept,  protection  evaluation  would  seem  to  consist  of  two 
subtasks: 
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1. 


“Normalizing"  the  target  system  by  extracting  the  information  relevant  to  the 
evaluation  and  representing  it  in  the  form  required  by  a comparison 
procedure. 


?.  Executing  the  comparison  procedure. 


Such  an  ideal  is  clearly  out  of  reach,  however.  There  exists  no  model  into  which 
the  protection-relevant  features  of  an  existing  system  can  be  mapped  and  in  which  they 
can  be  related  for  comparison  with  given  patterns,  general  enough  to  apply  to  wide 
classes  of  errors  and  systems.  It  is  even  difficult  to  determine  with  precision  which 
elements  of  existing  systems  are  relevant  to  protection  and  which  are  not. 


Nevertheless,  the  goal  of  developing  pattern-directed  techniques  arni  tools  to 
systematize  and  automate  protection  evaluation  might  be  achieved  with  a somewhat 
altered  approach.  This  becomes  evident  when  one  investigates  what  the  two  major 
requirements  for  protection  evaluation  techniques  imply  about  their  form,  application, 
and  development. 

Ihe  first  requirement,  that  of  general-purposeness  with  respect  to  operating 
systems,  carries  an  obvious  implication:  there  must  exist  some  generalized  set  of 
terminology-  a “comparison  language"— in  which  the  techniques  are  specified  and  in 
which  the  error  patterns  are  expressed.  To  apply  these  techniques  to  a given  system, 
it  is  necessary  that  a correspondence  be  established  between  Ihe  objects  and 
terminology  of  the  comparison  language,  i.e.,  between  Ihe  features  of  the  given  patterns 
and  their  instantiations  in  the  target  system.  Either  the  features  of  the  patterns  must 
be  instantiated  to  the  concepts,  objects,  and  terminology  of  the  target  system  or  the 
target  system  must  be  represented  in  terms  of  the  comparison  language,  or  an 
intermediate  comparison  framework  must  be  established  and  transformations  performed 
in  both  directions.  If  no  error  possibilities  are  to  be  overlooked,  then  all  the  instances 
of  a given  pattern  feature  in  the  target  system  must  be  identified. 

If  one  uses  the  term  "features"  to  refer  to  objects  that  have  concrete  and 
typically  localized  representations  in  the  target  system  description  (e.g.,  variables, 
procedure  calls,  critical  parameters),  then  identifying  the  relevant  features  in  the  target 
system  is  only  part  of  the  problem.  The  other  part  is  to  determine  whether  any  of  the 
relations  among  these  features  are  those  indicated  by  the  conditions  of  an  error 
pattern.  The  requirement  that  evaluators  need  not  have  a talent  for  recognizing 
protection  errors  and  that  difficult  pattern-recognition  processes  must  not  be  involved, 
makes  it  essential  that  the  search  for  an  error  be  decomposed.  The  search  through  the 
target  system  code  (or  some  representation  of  it)  for  a single  dispersed  collection  of 
instances  of  features  in  some  given  relation  must  be  replaced.  Instead  we  must  require 
only  independent  searches  for  individual  instances  of  features  in  the  target  system. 
1his  implies,  of  course,  that  the  output  of  these  searches  must  include  simple 
specifications  of  the  contexts  in  which  the  feature  instances  were  found.  The  needed 
feature  context  is  determined  from  the  relations  expressed  in  the  patterns  and  is  used 
to  determine  whether  the  features  found  actually  satisfy  these  relations;  Thus,  the 
single  integrated  search  step  is  replaced  by  a two-step  procedure,  the  first  of  which  is 
more  amenable  to  automation,  while  the  second  is  probably  best  performed  manually. 
While  the  analysis  of  the  relations  among  features  is  not  avoided,  it  is  deferred  to  a 
more  convenient  point  in  the  process  where  the  feature-set  to  be  consiefered  is  greatly 
reduced  in  size. 
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In  the  cnr.o  of  the  inconsistency  errer,  the  feature  extraction  process  was  applied 
to  a particular  instantiation  of  the  error  type  involving  the  consistency  of  user-auppllad 
parameters  in  the  MIILTICS  operating  system.  To  find  instances  of  the  error  in  code,  a 
pattern  was  formed  using  the  Error  Statement  above,  which  was  then  instantiated  for 
identifying  inconsistent  parameter  usage,  Error  Statement  requires  the  exiatenca 
of  two  operations,  both  of  which  refer  to  a common  variable  X.  The  first  Operation,  L, 
either  fetches  the  value  of  the  variable  or  gep<;rates  a hew  value.  The  secortd 
operation,  M,  fetches  the  value  of  the  variable.  Other  information  contained  In  the 
Error  Stalcmcnt  includes  the  fact  that  L occurs  before  M and  that  M performs  aome 
critical  function.  Those  statements  give  rise  to  the  following  pattern  elements: 

1.  An  operation  L which  either  fetches  or  stores  into  a cell  X. 

2.  An  operation  M which  fetches  cell  X. 

3.  Operation  Kf  is  critical. 

4.  Operation  L occurs  before  operation  M. 

For  this  particular  error,  X is  instantiated  to  a parameter,  and  thus  the  following 
additional  patlcrr*  element  is  derived: 

5.  A procedure  B which  is  interdomain-callable  by  user  procedures  and  which 
accepts  a parameter  X. 

This  pattern  ultimately  resulted  in  the  following  search  procedure  Intended  to 
recognize,  for  each  parameter,  executable  sequences  of  store  or  fetch  operations 
followed  by  a fetch  operation; 

1.  Filter  out  everything  except  procedures  which  are  interdomein-Cilleble  by 


2.  Of  these,  identify  those  with  parameters. 

3.  For  each  parameter,  identify  and  output  all  instructions  or  statements  which 
involve  store  or  fetch  operations  on  the  parameter. 

4.  Identify  and  output  all  instructions  or  statements  which  contain  flow  of  control 
operators. 

This  procedure  was  subsequently  automated  and  applied  to  MULTICS  with 
significant  success,  resulting  in  the  detection  of  a number  of  cendidate  errors  [BiS'^76]. 


COMPARISON  PIKK'.KSS 

The  search  output  constitutes  the  input  to  a separate,  methodical  comparison 
process  in  which  the  properties  of  the  feature  instances  found  are  exami:ied  to 
determine  whether  actual  error  conditions  exist.  Obviously,  the  comparison  is  still  not 
direct,  since  a translation  must  be  made  between  the  generalized  relations  expressed  In 
the  patterns  and  the  descriptions  of  feature  instances  provided  as  input.  Again,  In 


f: 

E t 
B m 


' ’ I jif  aiCTigpygfiHip 


PROJECT  DESCRIPTION 


11 


gftncral  the  choiro  must  be  made  between  expressing  the  search  results  in  the 
comparison  language  and  instantiating  the  reference  properties.  The  former  is  required 
for  a system-  independent  comparison  algorithm. 

In  the  case  of  the  inconsistency  error,  that  comparison  was  handled  manually. 
1 he  feature  matches  were  examined  manually  to  determine  if  the  second  operation  was 
in  fact  critical.  Forty-seven  procedures  were  examined  in  the  ML^TICS  system.  Of 
these,  seven  were  observed  to  have  one  or  more  errors;  five  other  procedures  had 
matches  for  which  "criticality"  of  the  second  fetch  could  not  be  determined  due  to  lacK 
of  system  documentation. 


.1.  RKDIHKCTIOS  OF  RKSK/IKCII 


In  Septcmb(jr  1975  the  research  direction  was  significantly  modified  to  conform  to 
revised  schedule  and  resource  considerations.  The  major  problem  with  the 
pattcrn-direclcd  approach  (detailed  analysis  and  relating  of  error  characteristic  from 
the  botlom-up)  was  that  the  process  was  both  time-consuming  and  extremely  tedious^  it 
consumed  a substantial  amount  of  the  project's  resources  while  yielding  few 
demonstrable  results.  The  sponsor  questioned  whether  or  not  the  protection  analycls 
process  was  bounded- -i.e.,  whether  the  number  of  error  categories  was  both  finite  and 
small  enough  to  warrant  the  expenditure  of  the  resources  required.  The  project  was 
asked  to  postulate  the  highest  level  error  categories  directly  from  the  existing  error 
data  base— to  categorize  the  entries  in  the  error  data  base  in  some  appropriate  fashion 
based  upon  the  analysis  performed  to  date.  We  were  to  subsequently  work  from  the 
postulated  error  categories  to  develop  automatable  search  strategies  rather  than 
pursue  the  pattern-directed  approach  of  gradually  building  up  a set  of  empirically  based 
categories.  It  was  thought  that  we  might  short-circuit  some  of  the  more  time-consuming 
elements  of  the  pattern-directed  approach,  directly  identifying  an  appropriate  sat  of 
error  types  without  having  to  devote  much  effort  to  analyzing  individual  errors.  The 
process  was  expecled  to  be  iterative,  possibly  leading  to  a sot  of  nonoverlapping  error 
calcgorics  which  could  bo  precisely  defined  ar>d  which  covered  the  known  protection 
vulnerabilities  in  existing  operating  systems  and  ultimately  to  viable  search  techniques 
for  identifying  instances  of  the  error  categories  in  target  operating  systems.  Thus,  the 
earlier  approach  as  characterized  by  Figure  2 was  supplanted  by  that  represented  In 
Figure  3 below. 
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Figure  3 


Various  difficuUicr.  were  encountered  along  the  way— unexpected  problems  which 
further  altered  our  approach  and  perspective  as  to  the  most  appropriate  strategy  for 
achieving  the  original  goals.  They  are  mentioned  below  in  the  discussion  of  ttte  specific 
steps  in  the  revised  process. 


REDIRECTION  OF  RESEARCH 


13 


KHROH  CATKCORI/AriON 

As  ^ consequence  of  Ihe  error-pattern  activities  the  errors  collected  in  the  error 
data-base  had  already  been  redescribed  in  a self-consistent  fashion.  Thus  an  attempt 
was  made  to  directly  identify  a set  of  categories  which  covered  the  recorded  set  Of 
protection  errors.  These  categories  were  to  serve  the  purpose  of  grouping  liKe  error 
types  for  in-depth  study  and  analysis.  The  expectation  was  that  the  categories  v/oulcs 
be  refined  as  the  analysis  process  proceeded  until  a final  set  of  highly  representative, 
nonintersecting  categories  was  identified. 

Ten  categories  were  identified  which  seemed  to  cover  all  the  errors  which  were 
documented  and  which  did  not  exclude  any  Known  error  types.  Unfortunately-  the  ten 
categories  seemed  to  manifest  themselves  at  differing  levels  of  abstraction;  thus,  it  was 
assumed  that  this  would  not  be  the  final  set  of  categories,  that  some  would  be  absorbed 
by  more  abstract  categories  or  possibly  be  a basis  for  new  categories  when  additional 
analysis  had  been  completed.  The  categories  are  briefly  described  in  Appendix  A. 


ANAI.VSIS  Oh  INOIVnniAL  CATECORIKS 

After  an  initial  set  of  categories  had  been  identified,  attention  was  directed 
toward  analyzing  individual  categories  to  gain  additional  understanding  into  the 
associated  operating  system  security  vulnerabilities,  allow  refinement  of  the  categories, 
and  accommodate  the  identification  of  search  techniques  for  given  error  types.  The 
categories  which  first  received  attention  were  those  which  appeared  lo  be  the  most 
tractable  and  manilcsled  themselves  at  the  less  abstract  levels  of  system  object 
representation.  The  error  type  "Inconsistency  of  a Single  Data  Value  over  Time," 
pursued  under  the  pattern-directed  work,  had  been  particularly  tractable  and  facilitated 
identification  and  implementation  of  specific  tools  for  identifying  errors  of  this  type  in 
existing  operating  systems.  The  results  of  our  efforts  on  that  error  type  suggested 
that  a quite  comprehensive  semi -automated  search  could  be  conducted  for  such  errors 
in  a given  operating  system.  It  was  hoped  that  the  same  would  hold  true  for  other 
error  types. 

Analysis  of  tho  second  error  category  led  to  a somewhat  different  result,  however.  In 
studying  tho  error  category  "Vaiidation  of  Operands”  it  became  apparent  that  the 
objects  under  consideration  were  much  less  tangible  than  those  dealt  with  in  the 
" Inconsistency. document.  The  definition  of  an  operator  or  operand  depended 
primarily  on  Ihe  level  of  abstraction  on  which  the  operating  system  was  being 
represented,  and  the  necessary  validation  was  generaliy  at  a comparable  level  [Carl76]. 

A general  strategy  was  devised  for  reviewing  an  operating  system  for  errors  of 
this  type,  and  the  requisite  tools  were  identified.  However,  the  analysis  of  this  error 
type  brought  into  sharp  focus  the  requirement  for  research  in  the  area  of  program 
verification,  since  Ihe  objectives  of  program  verificatio.n  and  the  requisite  effort  in 
diagnosing  errors  of  this  type  were  quite  similar.  With  this  error  type  it  became 
apparent  that  the  formalization  and  abstractions  that  were  part  and  parcel  of  verifying 
an  operating  system  were  also  important  in  identifying  points  where  validation  of 
critical  conditions  had  not  taken  place  or  had  been  implemented  improperly. 
Determination  and  analysis  of  Ihe  cumulative  effect  of  conditions  and  results  along 
relevant  control  paths  as  is  addressed  in  the  area  of  program  verification  is  also 
required  in  identifying  points  where  incomplete  validation  has  occurred. 
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lh(>  third  error  type  analyzed  was  that  of  residuals,  i.e.,  information  left  over  In 
an  object  when  the  object  is  deallocated  from  one  process  and  allocated  to  another. 
Residuals  reprosentod  the  first  error  type  which  had  a particularly  concrete 
manifestation  in  terms  of  operating  system  objects  (data  left  undestroyed  Ih  a 
deallocated  cell)  as  well  as  being  a highly  intuitive  error  type.  However,  it  WM  evident 
from  the  outset  that  the  causes  of  residual  errors  might  well  result  from  other  types  of 
errors  and  that  this  category  might  eventually  be  absorbed  by  one  or  more  categories 
handled  later  on  [Hol076].  A strategy  tor  identifying  sources  of  residual  errors 
amenable  to  partial  automation  was  identified  but  once  again  it  became  apparent  that 
iiucccsstul  identification  of  the  causes  of  residual  errors  in  operating  systems  would 
require  sophisticated  tools  involving  symbolic  program  execution  and  control  flow 
analysis  as  well  as  possibly  application  of  program  verification  techniques  in  order  to 
determine  the  paths  and  condition  sets  that  might  result  in  bypassing  of  code  intended 
to  clear  data  cells  on  deallocation. 

The  fourth  and  final  error  type  undertaken  was  that  of  serialization.  Treatment 
of  this  error  type  launched  the  project  into  consideration  of  the  fundamental  notions  of 
program  structure,  operator  synchronization,  principles  of  programming  practice,  etc., 
and  it  became  quite  difficult  to  identify  a viable  search  strategy.  As  a side  effect,  it 
became  immediately  evident  that  the  error  type  "Interrupted  Atomic  Operations"  was  a 
special  manifostation  of  this  error  category  and  should  be  treated  in  the  same  context. 

A niajor  consequence  of  work  on  the  aforementioned  error  types  was  that  It 
became  apparent  that  the  original  ten  error  categories  might  be  retorn.ulated  in  a more 
meaningful  way  in  terms  of  the  following  four  global  error  categories: 

1.  Domain  Errors 

2.  Validation  Errors 

3.  Naming  Errors 

4.  Serialization  Errors 

The  remainder  Of  the  ten  error  typos  (with  the  exception  of  the  operator 
selection  errors)  presented  earlier  seem  eithc'  to  fall  into  or  split  across  the  four  iypes 
shown  in  Table  1. 

Of  those  tour  categories,  two  (serialization  and  validation)  were  addressed 
explicitly  as  a result  of  the  woik  on  the  Ion  originally  hypothesized  error  types;  the 
other  two  (naming  and  domain  errors)  were  partially  covered  through  the  enalysis  of 
one  of  the  remaining  error  types  (allocation/deallocation  residual  errors).  However,  the 
bulk  of  the  examples  associated  with  the  latter  two  categories  have  not  been  addressed 
at  any  greater  detail  than  was  required  to  group  them  into  their  respective  categories. 
Thus,  while  we  believe  that  the  four  general  categories  end  their  respective 
subcategories  identified  represent  a useful  and  representative  grouping  of  example 
errors  and  a basis  for  more  directed  analysis,  it  is  possible  that  further  study  end 
analysis  would  result  in  an  even  more  insightful  error  classification  set. 

Appendix  B summarizes  the  four  documents  produced  by  the  project  which 
address  the  aforementioned  error  types. 
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4.  CONCimiONS  AND  FUlVHK  HKSKAHCll  DIHECTIONS 


tn  p^cncral,  the  technical  community  ha&  continually  underostimatoci  the  difficulty 
of  the  security  problem;  wo  feel  that  the  PA  effort  was  no  ©xceptlOh.  It  has  provad 
surprisinply  difficult  to  diagnose  protection  error  vulnerabilities,  much  lets  design 
techniques  for  deleclir^g  them.  However,  while  the  PA  projoct  1$  terminating  at  ISI  we 
feel  that  work  might  be  profitably  continued  in  the  original  area  of  pattern-directed 
protection  evaluation  despite  the  inherent  difiicultics.  This  approach  proved  quite 
successful  for  the  case  in  v/hich  it  was  taken  to  compklion  and  we  feel  that  It  should 
prove  equally  successful  in  others.  Progress  occurs  at  its  own  rate,  however;  research 
of  this  type  is  painfully  slow.  Much  thrashing  about  and  some  false  starts  must  be 
allowed  for  if  real  progress  is  to  be  made  in  this  difficult  research  area;  the  desire  to 
produce  useful  results  quickly  can  be  counterproductive  to  the  total  effort. 


a 


The  PA  project  has  had  its  principal  impact  in  extending  the  Knowledge  bese  end  ^ 

general  understanding  of  operating  system  protection  vulnerabilities,  relating  apparently  | 

unrelated  example  errors  In  forms  of  those  common  characteristics  which  result  In  a % 

security  vulnerability.  In  addition,  it  has  identified  some  general  procedures  which  will 

bo  valuable  in  detecting  future  security  system  vulnerabilities.  Finally,  the  PA  project  | 

has,  along  with  other  efforts,  made  the  user  community  increasingly  aware  of  the  i 

amount  of  effort  and  the  extensive  cost  involved  in  producing  a system  which  has  even  i 

a remote  chance  of  providing  a reasonable  degree  of  security  in  an  open  environment.  3 

Unfortunately,  it  has  also  become  apparent  that  the  commercial  sector  is  unwilling  to  | 

bear  this  cost  at  |ho  present  time  - that  there  is  no  apparent  commercial  market  for  ^ 

systems  with  the  drvriopmoni  costs,  reduced  performance  and  usage  and  environmental  3 

constraints  that  must  be  accepted  if  secure  processing  is  to  take  place.  Consequently,  I 

the  procedures  developed  by  this  project  will  probably  be  of  little  benefit  to  the  i 

commercial  sector  and  of  only  marginal  benefit  to  the  military  sector  at  this  time.  They  I 

will  find  application  only  when  we  decide  that  the  value  of  data  security  and  personal  \ 

privacy  are  greater  than  the  price  we  must  pay  for  secure  data  processing.  i 

The  analysis  of  identified  error  types  was  particularly  useful  in  identifying  some 
appropriate  research  and  development  activities  in  the  area  of  data  security,  I 

particularly  with  respect  to  the  types  of  tools  required  if  protection  evaluation  la  to  | 

become  automatable.  Tools  of  the  sort  described  in  the  "Data  Dependency  AnalytU" 

document  will  be  needed  in  much  of  the  evaluation  activity,  but  might  be  constructed  ao  i 

as  to  be  gencralizablc  across  systems  and  programming  languages.  j 

■5 

During  the  research  effort  one  thing  that  became  evident  was  the  role  of  program  i 

verification  techniques  in  delecting  operating  system  security  vulnerabilities.  It  is  hard  > 

to  sec  how  truly  definitive  statements  about  the  security  afforded  by  an  operating  | 

system  can  ever  be  made  until  PV  techniques  have  been  applied.  However,  certain  J 

unsettled  issues  about  the  appropriate  application  of  PV  techniques  to  0.$.  security  3 

analysis  suggest  that  research  in  protection  evaluation  might  bo  profitably  continued  In  t 

parallel  with  research  in  PV,  principally  to  insure  that  PV  is  applied  at  appropriate  1 

levels  of  operating  system  representation,  that  mapping  between  levels  Is  handled 
properly,  and  that  the  operating  system  is  represented  in  sufficient  detail  to  Insure  that 

security  vulnerabilities  do  not  go  undetected.  ^ 


fi 

B 


Ar.  9 final  footnote  to  this  research  effort  we  offer  the  following  comreent  for 
fhose  who  are  optim.otic  about  near-lorm  Improvement  of  the  data  security  problem. 
Our  insight  into  and  awareness  of  security  vulnerabilities  has  tended  to  vastly  exceed 
our  progress  in  detecting  and  correcting  them.  There  are  still  difficult  research 
problems  to  be  attacked  In  the  area  of  PE  in  particular  and  data  security  research  In 
general.  In  the  course  of  addressing  these  research  problems  there  will  undoubtedly 
be  much  floundering  and  some  abortive  starts.  Progress  can  bo  expected  to  be  painful 
and  slow  in  final  disposition  of  (he  security  problem^  particularly  since  such  work  seems 
to  Involve  delving  into  the  basic  premises  of  programming  theory  and  practice. 
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I.  CoM.M'.iIrtitoy  of  doio  otiflr  •inio 

Opcratin(\  sy^lome  continuously  mako  proloctlon-related  doclsions  based  on  data 
valuer,  contained  within  the  systom  data  base  as  well  as  on  values  which  have  been 
submitted  to  and  validated  by  the  system. 

In  order  for  a correct  protection  decision  to  bo  made  (In  Iho  absence  ot  other 
types  o(  proloction  errors),  the  data  must  be  in  a consistent  state,  and  remain  In  a 
specific  relationship  with  other  data  itents  during  the  interval  in  which  the  protection 
docision  is  made  and  the  corresponding  action  taken. 


2.  l^oltdotiou  of  opernnda 

Wittiin  an  opoi  ating  system,  numerous  operators  are  responsible  lor  maintaining 
the  system's  data  base  and  for  ctianging  the  protection  state  of  processes  Or  objects 
known  to  tho  sysirm.  M.any  of  these  operators  are  critical  In  Iho  sense  that  If  invalid 
or  unconstrained  data  arc  presented  to  thorn,  a protection  orror  results. 


.1,  Hexidunt* 


A generally  arcepted  error  type  Is  that  ot  the  "rcsiduaii"  l.e.,  Information  which  is 
"loft  over"  in  an  ohicct  when  the  obioct  is  deallocated  from  One  process  and  allocoted 
to  another.  Several  typos  of  residual  errors  exist,  including  the  following: 

1.  Across  residuals:  Incomplete  revocation  or  dc.allocation  ef  the  across 
cap.ibilitiot.  to  tho  object  or  coll. 

?.  Composition  residuals:  Incompleto  destructioi  of  the  cell’s  context  with  otlter 
colls  e;  ebiocis. 

9.  Data  residuals:  Incompleto  dostru'-fion  of  old  values  v ^thln  ‘he  cell. 


4,  Noming 

Names  are  used  within  operating  systems  to  dislinauist\  objects  from  one  another. 
There  arc  many  w.ays  in  which  name  binding  errors  can  load  to  protection  errors.  Tor 
example,  often  the  naming  scheme  does  not  have  enough  resolution  (or  does  not  use 
that  resolution)  to  distinguish  properly  belwoen  named  objects.  This  rese  ts  In  those 
errors  typified  by  a user  creating  an  ambiguity  by  naming  objects  vrith  ttie  same  name 
as  a previously  named  (or  about  to  be  named)  object  with  the  system,  ns  a result, 
rcfeioncing  the  wrong  object. 


Domnin 

A domain  is  an  authority  specification  over  an  object  or  set  of  objects  (usually 
thought  of  in  terms  of  an  address  space).  EnforcomonI  ot  domains  is  typical!/  limited  to 
ttio  resolution  of  the  hardware  protection  mechanism  provided  by  the  computer,  ktsny 
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Of  the  errors  In  operating  systoms  aro  tho  direct  result  ol  one  of  two  type*  of 
domdir>‘relatcd  errors: 

1.  Information  nssociatod  with  ttw  wrong  domain. 

?,  Incorroct  enforcemer^t  «l  domain  crossing. 


6,  Si'rhIixAlioii 

Within  any  oporating  system,  there  are  resources  to  which  the  operating  system 
must  not  only  control  access,  but  also  prevent  concurrent  use  or  otherwise  enforce 
orderly  use.  This  problem.  Known  as  “sorialization,"  is  ol  particular  Importance  In 
mulliprogramminp  systems  where  serialization  errors  often  result  In  protection  errors. 


7.  /Imiiiie  Oftornlimi 

Several  pi  election  errors  have  appeared  In  which  tho  enforcement  of  a 
protection  policy  was  basod  on  tho  assumed  uninterruplfibillty  of  an  operation.  In  each 
of  tho  cases,  tho  operation  was  in  tact  interruplable,  rosulting  In  a protection  error, 


ft.  perad  Kepra.mmelten.'i 

to  each  user,  an  operating  system  presents  an  abstract  machine  cor>sisti.ig  of  the 
hardware  user  instruction  sol  plus  the  psoudo-instructions  provided  through  the 
supervisor  catt/invor ation  mcchanisav  The  pseudo-instructions,  In  general,  allow  the 
user  to  m.inipulalc  abstract  objects  tor  which  representations  and  operations  are  not 
provided  in  tho  basic  hardware  instruction  set.  Inadvertent  exposure  by  the  system  of 
tho  representation  of  the  abstract  object,  the  primitive  instructions  which  Implamant  tha 
pseudo-instructions  or  the  data  structures  Involved  in  the  manipulation  of  the  abstract 
object  can  sometimes  result  in  protected  information  being  made  accessible  to  the  user, 
thereby  resulting  in  a protection  error. 


9.  Mnnngi'mrnt  /VporKfrnciea 

This  error  type  broadly  includes  those  errors  characterized  by  improper  or 
incomplete  handlini,  ol  boundary  conditions  in  manipulating  data  structures  Such  as 
system  queues  or  tables.  The  consequence  is  generally  a system  crash  or  lockup 
resulting  in  gross  denial  of  sofvico.  Wo  distinguish  this  from  IcgiJimate  denial  of  service 
conditions  when  the  system  is  merely  overloaded,  but  still  functioning  according  to  the 
scheduling  algorithm  design  specifications. 

10.  Crilicot  Ofit'rolar  Si'lrriion  F.rrort 

This  error  type  includes  those  errors  in  which  tho  impicmenter  invoked  the  wrong 
function,  statement,  or  insiruclion  resulting  in  tho  program  performing  the  wrong 
function,  In  a sense,  this  •$  a catch-all  category,  since  every  programming  error  can 
ultimately  bo  so  classified. 
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The  purpose  of  this  appendix  is  to  provide  a context  for  reading  the  respective 
error  detection  papers. 

/nr.oii.^intniiry  of  n nnalo  dnto  value 

A common  error  in  contemporary  operating  systems  is  the  assumed  consistency 
of  operands  between  multiple  uses.  If  an  operand  can  be  modified  between  two  uses 
by  a program  and  the  second  use  relies  on  an  attribute  referenced  in  or  set  by  the  first 
usage,  an  error  results.  Multiple  usage  of  a single  operand  often  occurs  during 
validation/usc  sequences  where  an  operand  is  first  validated  and  subsequently  used  in  a 
computation.  Numerous  variations  exist  that  make  locating  instances  of  the  error 
difficult.  For  example,  the  operand  can  be  referred  to  by  different  names,  or  the  uses 
may  be  contained  in  textually  disjoint  routines. 

Two  patterns  for  finding  inconsistency  errors  are  as  follows: 

la.  Find  any  sequence  of  REFERENCE  ...  REFERENCE  to  a common  operand, 
or 

1 b.  Find  any  sequence  of  STORE  ...  REFERENCE  to  a common  operand, 
whenever 

2.  the  operand  can  be  modified  between  the  pair  of  operators. 

Deteriion  of  luronxiiiieucy  Krrort.  Outlined  below  is  a set  of  search  strategies  for 
finding  consistency  errors  based  on  detecting  possible  instances  of  condition  la  or  lb. 
Largo  portions  can  be  automated. 

Consider  the  possible  storage  classes  that  operand  A can  take  with  respect  to  the 
routine  containing  the  two  references.  They  are  limited  to  one  of  the  following  three; 

1.  A local 

2.  A parameter 

3.  A global 

Case  1:  Local  Operand 

If  the  operand  is  local  (in  the  sense  that  no  other  routine  can  access  it),  then  the 
error  cannot  occur  and,  thus,  no  search  technique  is  needed. 

t*aramctcr  Operand 

If  the  operand  is  a value  parameter,  then,  since  it  is  copied  at  invocation  time  into 
a local  variable  within  the  routine  in  question,  it  can  be  treated  as  a local  operand  as  In 
Case  1.  If  the  operand  is  a name  or  reference  parameter,  the  following  search  strategy 
applies: 

1.  For  each  parameter  within  a routine,  find  all  reference  and  store  instructions 
to  the  parameter. 
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2.  For  the  routine,  find  all  control  flow  operators. 

3.  For  any  REFERENCE  ...  REFERENCE  or  STORE  ...  REFERENCE  on  a control  path 
(determine^.*  by  the  control  flow  operators  found  in  2),  examirw  the  pair  to 
determine  if  the  second  reference  operation  relies  on  an  attribute  referenoad 
or  stored  by  the  first  operator. 

4.  For  any  contiol  path  that  allows  a single  REFERENCE  to  be  executed 
iteratively,  determine  if  the  second  execution  of  the  RtFEKENCE  relies  on  an 
attribute  referenced  by  the  first  execution. 

The  above  procedure  finds  all  possible  occurrences  of  the  error  far  parameter 
operands.  Steps  1 and  2 can  easily  be  implemented  by  computer  program. 

Cajjc  3:  Global  Operand 


If  the  operand  is  a global,  then  it  can  be  accessed  by  multiple  routines.  The 
following  search  strategy  applies: 

1.  For  each  global,  find  all  reference  and  store  instructions  to  the  global. 

2.  Find  all  the  control  flow  operators. 

3.  For  any  Rl TERENCE  ...  REFERENCE  or  STORE  ...  REFERENCE  on  a control  path 
examine  the  pair  to  determine  if  the  second  reference  operation  relies  on  en 
atiribulo  referenced  or  stored  by  the  first. 

<1.  For  any  control  path  that  allows  a single  REFERENCE  to  bo  executed 
iteratively  or  recursively,  determine  if  the  second  execution  of  the 
RFFFRrNCF  relies  on  an  attribute  referenced  by  the  first  execution. 

Note  that,  with  one  exception,  this  is  the  same  search  strefegy  used  for 
parameters.  The  difference  is  that,  for  globals,  multiple  execution  of  a single  Instruction 
can  also  result  from  recursion.  Otherwise,  the  procedure  is  identical,  and  in  fact  the 
same  code  used  to  detect  potential  inconsistency  errors  for  parameters  can  also  be 
used  to  detect  potential  inconsistency  errors  for  globals. 

The  above  search  strategics  find  all  possible  consistency  errors.  A more  detailed 
description  of  Inconsistency  Errors  can  be  found  in  Bj5'f75. 


t 


A 


Valitinlion 

A 

Validation  of  operands  is  one  of  the  more  basic  functions  performed  in  operating  | 

systems,  it  constitutes  one  of  the  more  basic  error  types.  Validation  can  take  a variety  T 

of  forms,  from  checking  that  an  integer  subscript  is  within  the  bounds  before  allowing  i 

an  array  access  operator  to  proceed,  to  checking  that  a set  of  properties  such  as  the 

lime  of-day  and  the  caller’s  access  rights  hold  (or  an  operation  to  be  performed.  No 
single  evaluation  approach  seems  adequate  to  deal  with  the  wide  variety  of  validation 
found  in  conlomporary  systems  and  information  a protection  evaluator  may  have 
available  for  performing  the  evaluation  task.  As  such,  two  approaches  for  finding 

validation  errors  have  been  identified.  The  protection  evaluator  may  choose  either  or  a s 

combination  of  both.  J 
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The  first  requires  the  protection  evaluator  to  be  able  to  recognize  an  invalid 
condition  for  an  operand,  ft  begins  with  the  sources  of  data  needing  validation^  finds 
the  operators  which  use  such  data  (i.e.,  the.  which  are  potential  candidates  for 
validation  errors),  and  computes  the  valid  ^n  condition  holding  for  a given 
operator /operand.  A protection  evaluator  must  then  judge  the  adequacy  of  the  validity 
condition  for  the  given  operator.  The  second  approach  begins  with  operators  and 
validation  conditions  which  must  hold  and  determines  if  the  conditions  are  actually 
enforced  by  the  code.  It  requires  the  evaluator  to  be  able  to  identify  all  critical 
operators  and  specify  their  associated  validation  conditions  before  proceeding  with  the 
evaluation. 

OtM-u'ife-ie-fiwide  /IpproaeA.  A purpose  of  validation  is  to  prevent  privileged  system 
operators  from  operating  on  incorrect/unvalidatcd  operands.  Externally-supplied  user 
data  constitutes  such  a source.  They  enter  the  system  in  a variety  of  ways.  Direct  or 
indirect  parameters  to  supervisor  subroutines  constitute  one  large  source.  Others 
include  mutually  agreed  upon  mail  boxes,  communications  areas,  or  files.  The  operating 
system  is  responsible  for  insuring  that  this  data  is  properly  checked  before  a system 
operator  uses  it. 

One  approach  for  determining  the  adequacy  of  validation  is  to  begin  at  the 
user/system  interlace  and  calculate  the  validity  conditions  for  all  user-supplied  data  at 
various  operators  withit^  the  system.  This  can  be  done  as  follows: 

1.  Identify  all  data  entry  points  into  the  system.  (At  all  such  points,  data  can 
enter  the  system  that  needs  to  be  validated.) 

2.  f or  each  data  entry  point,  calculate  data  flow  paths  through  the  system.  All 
operating  system  variables  to  which  the  entering  data  is  directly  or  indirectly 
assigned  must  be  recorded. 

3.  Examine  all  operators  referencing  a variable  identified  in  (2)  above.  Verify 
that  the  validity  condition  enforced  on  each  data  path  leading  to  that 
operator /operand  is  sufficient. 

Step  2 can  be  automated  using  data  dependency  analysis  or  a modified  form  of 
symbolic  execution.  Steps  1 and  3 must  be  done  manually.  It  is  important  to  note-  'hat 
without  detailed  semantic  information  describing  operations  being  performed,  any 
procedure,  such  as  the  above,  can  only  tell  an  evaluator  where  to  look  for  errors,  but 
not  what  to  look  (or. 

Approach.  Suppose  a protection  evaluator  can  identify  all  critical 
operators  in  the  system  and  can  specify  for  each  operator  the  validity  condition  that 
must  hold  for  the  successful  completion  of  that  operator.  The  problem  of  finding 
validation  errors  then  amounts  to  determining  the  sufficiency  of  validation  code  on  all 
paths  leading  to  that  operator.  A procedure  for  checking  sufficiency  would  be  as 
follows; 

1 Identify  the  critical  operations  within  the  operating  system  and  the  necessary 
conditions  associated  with  those  operations.  Record  the  condition  with  the 
associated  operand. 

2,  If  an  operand  is  a local  or  a parameter,  follow  all  possible  control  paths 
leading  from  the  Operation  to  determine  the  data  paths  leading  to  the  critical 
operation.  In  passing  in  a reverse  direction  through  code  that  enforces 
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portions  of  the  validation  condition,  discard  the  enforced  corKlition, 
Eventually,  one  of  the  following  will  occur; 

a.  All  conditions  are  enforced  for  that  control  path. 

b.  All  conditions  are  not  enforced  upon  reaching  a user/system  interfecOi 

i.e.,  a validation  error  can  be  caused  by  supplying  a value  outside  the 
range  of  the  remaining  unenforced  condition. 

c.  The  control  path  terminates  at  a global  variable/parameter  interface 
within  the  system.  Go  to  3. 

3.  If  the  operanc  is  a global  or  format  parameter  from  2c,  all  operators  modifying 
the  global/p. irameter  must  contain  as  an  output  condition  the  validity 
condition  a';.ocialcd  with  the  respective  variables.  They  become  critical 
operators  to  be  evaluated  by  this  same  algorithm. 

A more  detailed  description  of  validation  errors  can  be  found  in  Carl76. 


Hrsidunh 

A common  security  problem  is  the  residual--data  or  access  capability  left  after 
the  completion  of  a process  and  not  intended  for  use  outside  the  context  of  that 
process.  If  a residual  becomes  accessible  to  another  process,  a security  error  may 
result.  A major  source  of  such  residuals  is  improper  or  incomplete 
allocation/deallocation  processing. 

Probably  the  most  widely  recognized  type  of  residual  is  the  data  residual  in 
which  some  property  of  the  data  associated  with  a cell  is  not  disposed  of  upon 
reallocation.  One  typically  thinks  of  content  residuals,  i.e.,  residuals  where  the  cell 
content  is  retnined  after  reallocation.  Data  residuals  can,  however,  involve  other  cell 
attributes.  Sucli  attributes  can  include  cell  size,  cell  location,  and  the  physical 
relationship  of  the  cell  to  other  cells.  While  not  representing  as  high  a communications 
bandwidth  as  the  content  residuals,  these  latter  forms  of  data  residual  can  also 
represent  significant  security  errors. 


The  following  procedure  for  finding  data  residuals  is  based  on  identifying  the  cell 
allocalion/doallocation  routine  in  which  residual  prevention  code  should  be  contained.  It  j 

consists  of  four  basic  steps:  ‘ 

; 4 

1.  Identify  all  cell  types  found  in  the  system.  This  can  be  done  by  manuatly  g 

listing  various  storage  inedia  and  cells  on  that  media  and  by  examining  system 

data  declarations.  ^ 

2.  For  each  cell,  identify  its  particular  freepool,  i.e.,  the  buffers  for  cell  | 

resources  between  deallocaMon  and  allocation.  | 

3.  For  each  freepool,  identify  allocation/dcallocation  code  by  finding  all  symbolic  1 

references  to  the  freepool.  j 

4.  For  each  allocation/dcallocation  routine,  determine  if  a data  residual  can  | 

occur.  4 

f. 

1 
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A socond  m.tjor  type  of  residual  is  the  access  management  residual)  sometimea 
Known  as  a ''d.ingling  toferer.ee.**  Unlike  data  residuals  that  deal  with  the  various 
attributes  of  a coll,  access  management  residuals  deal  with  the  access  paths  used  to 
reference  a cclli  their  creation  and  destruction. 

Access  paths  are,  at  some  level  of  representation,  simply  data  stored  in  special 
colls  (c.R,,  bounds  registers,  PSW's,  segment/page  tables,  capability  cells,  etc.).  Thus, 
techniques  sir^iilar  to  those  described  above  lor  finding  content  residuals  will  also  find 
certain  lypc^  jI  access  residuals,  i.c.,  those  caused  by  incomplete  deallocation  of  an 
access  path  created  by  an  allocation  routine.  Access  management  residuals  differ  from 
content  residuals  in  an  important  aspect.  There  may  be  multiple  access  paths  to  a 
given  cell,  all  of  which  must  be  deallocated.  Furthermore,  access  paths  can  be  created 
by  other  ttian  the  formal  allocation  routines.  For  example,  code  that  copies  an  existing 
access  path  produces  an  access  path  which  must  also  be  accounted  tor  at  deallocation. 
Similarly,  special  instructions  may  exist  (e.g.,  the  IBM  370  **LOAD-REAL-ADDRESS")  that 
produce  access  paths  as  a rc'sult  of  invocation,  or  that  can  be  interrupted  causing  an 
access  path  to  be  stored  for  use  when  the  instruction  is  reinvoked.  Thus,  in  addition  to 
the  above  procedure,  one  must  examine  the  system  for  these  latter  three  sources  of 
access  paths  and  account  for  the  paths  at  cell  deallocation, 

A more  detailed  description  of  Residual  errors  can  be  found  in  HolB76. 


Srriplittilion 

Serialiralion  errors  represent  one  of  the  broader  categories  investigated.  As 
such,  the  error  has  numerous  manifestations  and  can  bo  described  in  a variety  of  ways 
including  ordering  opccificationsi  interoperation  communic.stion  and  insuring  the  proper 
use  of  communication  channels;  mutual  exclusion  tor  preserving  object  integrity;  and 
mutual  exclusion  (or  the  noninterference  of  non-atomic  operations. 

Three  distinct  approaches  for  detecting  scriatization  errors  are: 

1.  An.ily/e  the  target  system  macroscopic  ally  and  informally  for  the  adequacy  of 
each  ol  a list  of  serialization  provisions.  The  problem  with  this  approach  Is 
that  no  actual  algorithm  is  suggested  by  the  serialization  provisions  lor 
dc'ciding  when  serialization  errors  do  or  do  not  exist. 

2.  Determine  potential  concurrencies,  and,  given  these,  determine  whether  any  of 
them  (laKen  pairwise)  represent  access  conflicts. 

3.  Assume  all  access  sequences  to  sharabic  objects  are  critical  and  represent 
potentially  conflicting  concurrencies  unless  these  are  made  impossible  either 
by  explicit  invocations  of  serialization  mechanisms  or  by  other  serializing 
program  logic.  The  problem  with  this  approach  is  that  it  detects  a great 
mai>y  access  intervals  that  are  not  serialized  in  an  obvious  manner,  artd  one 
must  then  resort  to  deeper  analysis  such  as  that  in  (2). 

Each  approach  Is  discussed  in  greater  detail  along  with  suggested  ways  for 
alleviating  deficiencies  in  Carl  78. 


